DOCUMENT RESUME 



ED 452 820 



IR 020 657 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Penuel, William R.; Means, Barbara 

Observing Classroom Processes in Project-Based Learning 
Using Multimedia: A Tool for Evaluators. 

1999-00-00 

15p , ; In: The Secretary* s Conference on Educational 
Technology, 1999: Evaluating the Effectiveness of 
Technology. [Proceedings] (Washington, D,C., July 12-13, 
1999) ; see IR 020 654. 

For full text: 

http : //www . ed . gov/ technology/1999/whitepapers . 

Reports - Research (143) -- Speeches /Meeting Papers (150) 
MF01/PC01 Plus Postage. 

♦Classroom Observation Techniques; Computer Uses in 
Education; Educational Technology; Evaluation Methods; 
Learner Controlled Instruction; +Multimedia Materials; 
♦Observation; *Program Evaluation; *Student Evaluation; 
♦Student Projects 



ABSTRACT 



This paper describes how researchers at SRI International 1 s 
Center for Technology and Learning designed and used an observation tool as 
part of its evaluation of a local Technology Innovation Challenge Grant 
program called Challenge 2000: Multimedia Project. The paper aims to present 
both the process and the tool that researchers, program designers, and 
teacher-researchers might adapt to similar programs and initiatives. Methods 
are discussed for observing changes in classroom processes in project-based 
classrooms using multimedia technology. The design of the observation tool is 
discussed and findings are presented about the differences in classroom 
processes between Multimedia Project classrooms and comparison classrooms. 
Findings indicated that Multimedia Project classrooms were more likely to be 
learner-centered and to engage students in long-term, complex assignments. 
Overall, the results suggest that the project is meeting its objective of 
transforming classroom processes so that they become more student -centered, 
especially while students are engaged in project-based learning using 
multimedia. (Contains 21 references.) (AEF) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



IR020657 ED 452 820 



Observing Classroom Processes in Project-Based 
Learning Using Multimedia: A Tool for Evaluators 



a Minor changes have been made I o 
improve reproduclion quality. 



doiumen mT °'. opi " ions slated Tri ih'is 

o d S e o n ^"°, nOC - S -^™P-3on1 
omcrar ubRI position or policy. 



By: William R. Penuel & Barbara Means 



o 

ERLC 



2 BEST COPY AVAILABLE 



Observing Classroom Processes in Project-Based Learning Using Multimedia 

The Secretary's Conference on Educational Technology- 1999 



Observing Classroom Processes in 
Project-Based Learning Using 
Multimedia: A Tool for Evaluators 

William R. Penuel 
Barbara Means 

Center for Technology in Learning SRI International 



Abstract: This paper discusses methods for observing changes in classroom processes in 
project-based classrooms using multimedia technology. The tool was used as part of a 
five-year evaluation of a local Technology Innovation Challenge Grant program called 
Challenge 2000: Multimedia Project. In the paper, we discuss the design of the observation 
tool and present findings about the differences in classroom processes between Multimedia 
Project classrooms and comparison classrooms. Project classrooms, we found, are more 
likely to be learner-centered and engage students in long-term, complex assignments. 

INTRODUCTION 

Case studies such as from Apple's Classrooms of Tomorrow (Sandholtz, Ringstaff, & Dwyer, 1996) and 
from reforming schools across the country (Means & Olson, 1995) point to the potential of new 
technologies to support new ways of teaching. Moreover, these studies have provided rich details about 
what takes place in classrooms as local technology-supported reforms are implemented. 

In addition to these kinds of studies of technology use in schools, in recent years there have been a 
number of published reports using national data that show the promise of technology to support school 
reform. Many of these studies rely on survey data, student achievement data, or a combination of both in 
their analysis. Becker (1999), for example, has published some of the results from his recent survey of 
teaching practice and technology use in the United States on the World Wide Web, in which he found 
fewer than 1/3 of teachers were having students conduct research on the Web. The Educational Testing 
Service has found that when students use computers to apply higher order concepts and when teachers 
are knowledgeable about how to use computers as productivity tools, students show significant gains in 
mathematics achievement (Wenglinsky, 1998). 

As initiatives and programs that use technology to help drive school reform proliferate, there is an 
increasing need for tools that can help measure whether these programs achieve anticipated changes in 
teaching practice. Certainly tools used in both case study research and large-scale studies can be used or 
adapted for use in evaluation, but researchers must pay careful attention to local program contexts when 
considering how to use these tools. First, evaluators must first examine the program or initiative's 
specific design; that is, how interventions are expected to bring about particular changes in teaching 
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practice. The anticipated changes must themselves be described in enough detail for researchers, other 
observers, and teachers themselves to be able to know when they've achieved those desired changes in 
teaching practice. Second, evaluators must consider a range of factors related to the opportunities and 
constraints of the evaluation process itself: What is the scope of the evaluation? Who are the stakeholders 
and partners in the evaluation? What kinds of data are needed to evaluate the design? How will the data 
be analyzed? How will different partners in the initiative use the data? 

In this paper, we describe how researchers at SRI International's Center for Technology and Learning 
designed and used an observation tool as part of its evaluation of a local Technology Innovation 
Challenge Grant. The paper is intended to present both a process and a tool that researchers, program 
designers, and teacher-researchers might adapt to similar programs and initiatives. First, we describe both 
the program's theory of action and some of the important opportunities and constraints in the evaluation 
design. Next, we describe the design of the observation tool itself and how it was used over the course of 
two years. Finally, we present some of the key findings from the observation study, with specific 
attention to changes in teaching practice that were observed. 

THE CHALLENGE 2000: MULTIMEDIA PROJECT DESIGN 

The context of this study is a project funded through the U.S. Department of Education's Technology 
Innovation Challenge Grants program. The grant was awarded over four years ago to the San Mateo 
County Office of Education and is jointly coordinated by the county office and Joint Venture: Silicon 
Valley, a partnership of area businesses focused on improving the quality of life in the Silicon Valley 
region in northern California. The federal grant funded the Challenge 2000: Multimedia Project and was 
designed to support Joint Venture's larger education reform initiative, which is aimed at making Silicon 
Valley students among the most sought-after by employers in the region for jobs in the new workplaces 
of the twenty-first century. 

The Challenge 2000: Multimedia Project aims to engage students in their own learning and develop 
students' skills of collaboration, decision-making, and complex problem solving. To accomplish these 
goals, the Project has adopted a model of Project-Based Learning using Multimedia (PBL+MM) and 
provided supports to teachers in learning how to implement projects and use technology effectively to 
enhance and support student learning. By implementing student-centered projects and providing supports 
to teachers, it is expected that classroom processes and teaching practice will change, leading to better 
outcomes for students. 

Student-Centered Projects 

The model of project-based learning using multimedia is a research-based model developed by the 
Challenge 2000 participants in collaboration with researchers from the Institute for Research on 
Learning. This model incorporates all of the dimensions that have been traditionally associated with a 
project approach to learning (Blumenfeld, et al., 1991; Kirkpatrick, 1918; Rawcliffe, 1925), such as 
having a real-world connection, but adds the practice of producing final projects in a multimedia format 
as a central part of the practice. Among the kinds of multimedia products that students have produced are 
HyperStudio stacks, Web pages or sites, PowerPoint presentations, animations and videos, and music 
CDs. 

There are seven components of the Project Based Learning Using Multimedia model. Projects are 
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expected to: 

• Be anchored in core curriculum; multidisciplinary 

• Involve students in sustained effort over time 

• Involve student decision-making 

• Be collaborative 

• Have a clear real-world connection 

• Use systematic assessment: both along the way and end product 

• Take advantage of multimedia as a communication tool 

It is important to note here that the seventh component, use of multimedia technologies, is not conceived 
as a stand-alone component. Multimedia technologies are intended to be used as tools in the planning, 
developing, and presenting projects. It is believed that the power of multimedia lies primarily in the 
extent to which it is integrated within the goals of the project and ongoing curriculum for the class. 
Products that students create come to serve as public artifacts (Allen & Pea, 1992; Blumenfeld, et al., 
1991; Penuel, Cole, Korbak, & Jump, under review) that are part of the classroom community's memory 
of what it has accomplished. 



Teacher Supports for Technology Use 

To help teachers implement the PBL+MM model in their classrooms, Multimedia Project staff have 
created a number of supports and incentives for teachers. The project's theory of action has emphasized 
the importance of creating a learning community among participating teachers. Initially, the Institute for 
Research on Learning provided training on how to plan and implement projects in the classroom and on 
how to use multimedia technology. As the project developed, participating teachers formed a cadre (a 
Project name) that took on more and more responsibility for planning and conducting their own 
professional development. As teachers have become more practiced and skilled in implementing the 
model, they have refined it and shared it with new participants that join the Project. 

Multimedia Project teachers establish a peer community of learners in which they gradually take on 
responsibility for planning and conducting their own professional development. Veteran teachers share 
their skills with less experienced colleagues. Many of these veteran teachers serve in special roles funded 
in part through the Multimedia Project. Technology Learning Coordinators in the project are skilled in 
both the use of technology and innovative teaching practice and are available to teachers in the project 
for help. Typically, a portion of the Technology Learning Coordinators' time is also spent providing 
technical assistance to teachers experiencing problems with specific technologies. 

The Project also provides a system of recognition and rewards for project teachers. Teachers may apply 
individually or as partners for mini-grants, allowing them to purchase equipment, software, peripherals, 
and/or training if they implement multimedia projects in their classes. Providing greater access to 
hardware, software, and to the Internet has made it possible for students to complete projects they would 
never have been able to do before the grant. Teachers may also be recognized for their students 
contributions to annual Multimedia Fairs held by the school teams participating within the Challenge 
2000: Multimedia Project. 
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In the Multimedia Project design, the implementation of student-centered projects and development of a 
peer learning community with access to technology and technical support are expected to result in 
changes in what goes on inside project classrooms. Specifically, the design calls for classrooms in which: 

• students engage in longer-term, more complex assignments 

• teachers act as coaches and facilitators of student learning 

• students engage in more small-group collaborative activities 

• there is greater involvement with external resources, including heightened attention to external 
audiences for student work 

In turn, these changes in classroom practices are expected to bring about the student outcomes described 
above-greater skills in collaboration, decision-making, and complex problem solving. 

The changes in classroom processes can be seen as intermediate outcomes of the Multimedia Project, 
benchmarks that can be used to indicate that the project is progressing toward meeting its objectives for 
student learning. Because of the central importance of changed classroom processes and teaching 
practice in the program design, we chose to develop an instrument to help measure whether in fact 
participating classrooms were in fact more student-centered, collaborative, and engaged with external 
resources. In the next section, we describe the protocol and the evaluation context in which this 
instrument was designed. 

DESIGNING AN OBSERVATION PROTOCOL TO MEASURE 

CLASSROOM PROCESSES 



The Evaluation Context 

SRI International (SRI) is under contract to conduct the evaluation of the federal grant coordinated by 
Joint Venture: Silicon Valley and the San Mateo County Office of Education. The evaluation is a 
multi-method, five-year study of the implementation and outcomes of the Multimedia Project. As the 
project has developed over its first four years, the evaluation has moved from a primary focus on 
documenting implementation (describing student-centered projects and teacher supports) to measuring 
outcomes of the project. Research questions evolved as the project model has crystallized: each year, 
evaluators are able to ask more focused questions about teaching and learning in Multimedia Project 
classrooms. 

Throughout the project, SRI International has adopted a partnership approach to the evaluation process. 
Staff from SRI serve on the Multimedia Project's Coordinating Committee, which meets monthly to 
discuss the progress of the Project's activities. The Committee consists of representatives from San 
Mateo County Office of Education, JVSV, the Institute for Research on Learning, and other key program 
partners. At these meetings, SRI presents information about how teachers and students are participating 
in and responding to various project activities. Other members of the Coordinating Committee, led by 
JVSV, identify their own questions that in turn shape each year's evaluation design. 

SRI has used case studies, interviews, teacher surveys, classroom observations, school-wide indicators of 
achievement, and performance assessment data as part of the study. Each of these methods has been used 
either to document implementation of the project or measure progress toward outcomes. 
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Year 3 Study Findings 

Initially, the observation protocol was adapted from one used by researchers at the National Center for 
Research on Evaluation, Standards, and Student Testing (CRESST) for use in the studies of Apple 
Classrooms of Tomorrow in the third year of the program in part to provide evidence to program 
stakeholders that the Multimedia Project was making progress toward changing classroom processes in 
project classrooms. 

In this first study, 19 classrooms were chosen from among Challenge 2000 classrooms across grade 
levels for observation in the fall of 1997 and the spring of 1998. Principals from schools where SRI was 
conducting case studies nominated three technology-using and three non-technology-using classrooms 
for participation. In most cases, these schools had three teachers participating in the project, or otherwise 
engaged in technology use, but some did not. In those cases, additional non-technology-using classrooms 
were observed for the study. The original observation protocol examined variables such as the dominant 
classroom activities, teacher and student roles, the nature of ongoing student work, and the level of 
student engagement. 

The results of the study showed significant changes in classroom processes from fall to spring, with 
differences between technology-using and non-technology-using classrooms (Means & Golan, 1998). 

For example, in the fall students in technology-using classrooms were only slightly more likely than 
students in comparison classes to be engaged in long-term projects at the time of the observation. By 
spring, that gap was very wide, with 67% of technology-using classrooms versus 14% of non-technology 
using classrooms in volved in extended projects at the time of observation. Similarly, teachers from both 
sets of classrooms were equally likely to be engaged primarily in questioning students, a traditional role 
for teachers, in the fall. In the spring, far fewer technology-using teachers used questioning as their 
dominant way of relating to students (7% versus 49% for non-technology-using teachers). Instead, 
technology-using teachers were much more likely to be in a helping or monitoring role within the 
classroom (43% in the spring versus 18% of non-technology-using clas srooms). 

Similarly, students in technology-using classrooms were much more likely than their peers in 
non-technology-using classrooms to be engaged in constructing products and working in small groups in 
collaborative activity. Again, the differences were much greater in the spring than in the fall. In the fall, 
56% of technology-using classrooms involved students in constructing products compared to 39% of 
non-technology-using classrooms. By the spring, that gap widened: 73% of technology-using classrooms 
engaged students in constructing products versus 38% of non-technology-using classrooms. While in the 
fall, few classrooms from either sample engaged students in small-group collaboration, nearly a quarter 
of technology-using classrooms involved small-group collaborative activity in the spring (compared to 
0% of non-technology-using classrooms). 

Adaptation of the Protocol for Year 4 

A multi-year evaluation affords the opportunity to revise instruments and processes based on what is 
learned from using them and based on the purposes the instruments serve in the overall evaluation design 
for that particular year. At the beginning of Year 4, we made some revisions to the observation protocol 
itself and to the data collection and analysis process. 

In Year 4, the evaluation data collection activities focused increasingly on measuring outcomes from the 
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Multimedia Project. A performance assessment task was created to measure student skills in design, 
collaboration, and mastery of content. We decided to link the planned replication of the observation 
study with the performance assessment task. In this way, we could test the design or conceptual 
framework of the model by answering the question: Do changes in classroom processes lead to different 
levels of student performance? At the time of writing, performance assessment data are still under 
analysis. In this paper, we describe the design changes to the observation protocol and report results from 
Year 4, which continue to point to the promise of the design in changing classroom processes. 

Selection of Classrooms 

In Year 4, the classrooms were selected using a different method in order to ensure that a large number of 
veteran teachers would be included in the sample of Multimedia Project classrooms. For this reason, we 
do not use the terms technology-using and non-technology-using classrooms to characterize the two 
samples (though the two samples of classrooms can be distinguished in this way). Rather, the study is 
comprised of 12 project classrooms and 9 comparison classrooms. As in Year 4, observations were 
conducted once in the fall and in the spring, both times within a three-week window. 

Project classrooms selected for the study were a combination of experienced and novice teachers within 
the Multimedia Project who were funded with mini-grants for the 1998-99 school year. Principals from 
the project teachers' schools selected comparison classroom teachers. Principals were given instructions 
to select a teacher in the same grade who was not a part of the project but who taught in a subject area 
similar to the project teacher. Because the project encouraged partnerships within schools, finding a 
comparison teacher at the same grade level was not always possible. In two cases, classrooms from the 
same grade level at a comparable school in the same district were chosen. Still, the resulting classrooms 
were similar in size and in demographic composition (Table 1). 

Table 1. 



Composition of Classrooms in the Study 





MM Project Classrooms 


Comparison Classrooms 


Average attendance 


27.5 % 


28.4 % 


Ethnic composition 




White 


56 % 


61 % 


Asian/Pacific Islander 


20% 


17 % 


Latino 


15% 


16% 


African American 


2% 


4% 


Other 


7% _ 


2% 



Classrooms did differ on one significant measure, namely the number of computers that were in their 
classrooms. On average, Multimedia Project classrooms had 6 computers, while comparison classrooms 
had only 2. 

In addition, both samples included only 6^ and 7^ grade classrooms. We selected classrooms from these 
two grades because the performance task designed to measure the impact of the project on student 
learning was targeted to middle grades students. 
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Addition of Items to Protocol 

In Year 4, we added two sets of items to the protocol that have been emphasized by sociocultural 
researchers (Cazden, 1988; Lemke, 1985; Mehan, 1979; Wells, 19xx; Wertsch, 1991) as important for 
sustaining extended student inquiry. We asked observers to characterize the different forms of discourse 
that students and teachers used in the classroom. For example, observers looked for "instructional 
questions" (Mehan, 1979; see also Heath, 1983) in which teachers ask brief questions of students, to 
which the answer is already known, to test students knowledge of isolated facts. In general, we were 
interested to know whether Multimedia Project classrooms engaged in what have been called more 
dialogic (Bakhtin, 1981) forms of discourse than comparison classrooms. By dialogic, we mean forms of 
discourse that engage students and teachers in discussions that are not always teacher-controlled. By 
contrast, we anticipated that comparison classrooms might be more likely dominated by a monologic or 
lecture-oriented form of discourse. 

We also wanted to be able to analyze better the extent to which teachers allowed students to work 
independently with limited strategic assistance (Wertsch, 1985). We expected teachers in project 
classrooms be more inclined than those in comparison classrooms to allocate more time than comparison 
classrooms to having students practice learning skills on their own, rather than simply demonstrating the 
skills to students or telling them about what they need to know. We predicted that teachers would provide 
assistance as needed in project classrooms, but students would be given primary responsibility for their 
own learning. 

Activity as Unit of Analysis 

Consistent with a sociocultural approach to observing classroom practice, we also chose to use activity 
rather than observation time as the primary unit of analysis for our Year 4 observations. For purposes of 
the study, an activity is defined as student engagement in some kind of educationally relevant product. 
Those products include: a story written, a reading completed, a topic discussed, science observations 
made and recorded, a set of related problems at the board worked through, a pre- writing activity 
completed, a painting painted, et cetera. Sometimes an activity produces no tangible product (students 
listen to a lecture) but the activity is nonetheless organized to produce a definable outcome-e.g. coverage 
of a particular topic. 

Operationally, we defined the activities as different when two or more of the following changes took 
place in the classroom: 

• A new product or objective is introduced by the teacher or other students that is followed by new 
patterns of thinking, communicating, and acting. 

• The topic changes, whether signaled by movement from one subject area to another or to a 
different domain within a particular subject. 

• The activity or participant structure changes; in other words, the way roles are assigned among 
students or the ways students and teachers are interacting shifts (e.g., from whole group lecture to 
small group collaboration). 

• The spatial arrangements in the room shift, in that either people change places or physical objects 
in the room are re-configured to afford a different kind of activity. 

• The teacher (or students) makes a bid to close a segment of classroom activity, signaled by specific 
instructions to students about "wrapping up" or by teachers beginning to review just finished work 
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or instructing students about re-arranging space in the classroom. 

Multiple Levels of Analysis 

By selecting activity as the unit of analysis and recording the amount of time spent on each activity, 
additional avenues for data analysis were opened up. Whereas in Year 3, observers recorded what was 
happening in three fifteen-minute intervals throughout their observations, in Year 4, observers recorded 
anywhere from between 1 and 4 different activities across a forty-five minute observation period for each 
classroom. The Year 4 data permit analysis of the amount of time spent in various activities, by the 
dominant activities within classrooms, and by classroom teacher. In this paper, we present analyses from 
the first two dimensions, time and activity. 

OBSERVED DIFFERENCES BETWEEN PROJECT AND COMPARISON 

CLASSROOMS 

In this section, we report a number of significant differences we found between Multimedia Project 
classrooms and comparison classrooms. We consider the extent to which project classrooms were more 
focused about long-term projects, learner-centered, collaborative, and oriented toward people and 
communities outside the classroom. 

Engagement in Long-Term, Complex Assignments 

Time Spent on Long Term Assignments. Students in Multimedia Project classrooms engaged in 
significantly longer activities (p<.05) than students in comparison classrooms. Moreover, they were more 
likely to be engaged in long-term activities-that is, activities that spanned more than a week of class 
time-than their counterparts in the comparison classrooms. Moreover, both in the fall and the spring, 
students spent more time in project classrooms engaged in long-term activities that lasted a week or more 
(an average of 84% of the time in project classrooms versus 49% of the time in comparison classrooms). 

Analysis of the Complexity of Activities. For the Multimedia Project to be successful, it would not be 
enough to say that students are engaged in long-term activities. Something of the quality of their activity 
would need to be observed and understood to argue successfully that projects were transforming 
classroom processes. An analysis of student actions reveals that in fact, students are engaged in complex, 
cognitively challenging tasks in project classrooms. 

Students were observed to be engaged in more of what might be called the cognitive activities of design. 
In other words, they were engaged in the kinds of higher-level cognitive activities characteristic of 
multimedia design as described by Lehrer (1993): deciding on the structure of a presentation; creating 
multiple representations, models, and analogies; arguing about or evaluating information; thinking about 
one's audience; and revising or editing work. Table 2 shows the differences between Multimedia Project 
and comparison classrooms for those activities marked "dominant" by observers. 



Table 2. 

Dominant Activities Observed 





MMP Classrooms 


Comparison 






Classrooms 



10 



‘ ttp://www.od.gov/Technology/TechConf/1999/whitepapers/paper3.html (8 of 13) [4/24/2001 2:38:13 PM] 



Observing Classroom Processes in Project-Based Learning Using Multimedia 



Teacher-directed solo 
activities (e.g., reading 
silently, listening to teacher) 


13 


23 


Cognitive activities of design 
(e.g., deciding on structure of 
a presentation) 


13 


3 



c2=9.03,df=l,p<.01 



Teachers as Coaches and Facilitators 

Time Spent in Independent Activity. In Multimedia Project classrooms, more time was spent having 
students practice skills on their own (whether independently or as a group) with strategic assistance 
provided by teachers as needed, than having students watch or listen as teachers performed a task for 
them or explained a process to them (See Figure 2). This difference was particularly pronounced in the 
spring, when teacher-led activities comprised 29% of time in project classrooms versus comparison 
classrooms (62%). It is clear from these data that project teachers are more likely to give major 
responsibility to students for their own learning than do comparison teachers. 

Dominant Roles of Teachers. Teachers in Multimedia Project classrooms were much more likely to be 
engaged in facilitative roles within classroom activities than were teachers from comparison classrooms. 
In other words, they were more likely to be engaged in assisting or helping students by moving about the 
classroom and responding to student questions or providing help when they see a need for it. This 
facilitative role is evident in the greater extent to which teachers help to organize the process by which 
students can work productively on their own, whether in groups or individually. By contrast, the 
dominant role of teachers within comparison classrooms was more directive. Teachers were more likely 
to be explaining concepts, providing information, or questioning students about their understanding of 
material (see Table 3). 



Table 3. 

Dominant Teacher Roles Observed 





MMP Classrooms 


Comparison Classrooms^ 


Directive Role 

(e.g., explaining concepts, providing 
information, questioning students) 


11 


17 


Facilitative Role 

(e.g., assisting or helping, managing 
the organization of the task, 
monitoring as students work on their 
own) 


13 


7 



11 
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c2=7.81, df=l, p<.05 

Engagement in Small-Group Collaborative Activity 

Time Devoted to Small Group Activity Students in Multimedia Project classrooms were more likely than 
comparison students to spend time engaged in small group collaboration. This collaboration was 
supported, moreover, by discourse patterns that allowed students to direct discussion among their peers 
about the content of the class. 

While in the fall, students spent roughly the same amount of time in project and comparison classrooms 
engaged in small-group discussion, by the spring, project classrooms devoted much more time to this 
form of discourse. A corollary finding is that by the spring time, only 3% of the time in project 
classrooms was devoted to "instructional” or known-answer questions compared to 72% of the time in 
comparison classrooms. 

Analysis of Dominant Activities. An analysis by activity yields similar results. There was a more dialogic 
pattern of discourse within project classrooms than within comparison classrooms in the spring. By 
dialogic, we mean forms of discourse that engage students and teachers in discussions that are not always 
teacher-controlled (e.g., lecture). By contrast, comparison classrooms were much more likely to be 
observed as having a monologic or lecture-oriented discourse dominate classroom time (c2=7.88, df=l, 

p<.01). 

Involvement with External Resources 

Time Spent Using the Internet. One of the most valuable tools for connecting classrooms to wider 
communities is the Internet. By the spring, students in project classrooms spent half of the time observed 
using the Internet, searching for information, graphics, pictures, sounds, and other material to use for 
their multimedia presentations. The Internet was not used at all in comparison classrooms, either in the 
fall or the spring. 

Attention to an External Audience. Yet another way that classrooms are connected to broader 
communities is through the student-led projects themselves, which typically have an audience outside the 
classroom. In this respect, project classrooms differ significantly from comparison classrooms in the 
likelihood that students will be engaged in discussion about how their audiences would respond to 
aspects of a product being produced (Fisher Exact Test, p<.001). In spring, 35% of the activities in 
project classroom involved students considering the audience of their work, whereas none of the 
activities observed in comparison classrooms found students attending to the audience of their work 
(beyond the teachcr-as-audience). 

Discussion and Implications 

In each of the dimensions we observed and analyzed, Multimedia Project classrooms distinguished 
themselves from comparison classrooms by being significantly more student-centered and organized 
about the collaborative construction of complex products. These findings not only constitute evidence of 
the projects success in stimulating desired changes at the classroom level at classroom, but also measure 
power of our instrument to measure and capture these changes. 

At the same time, this effect was not evident throughout the whole school year on all dimensions. As in 
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Year 3, classrooms were much more likely to be engaged in focused efforts to complete student projects 
in the spring than in the fall, even though care was taken to select only Multimedia Project teachers for 
the sample. The only dimension in which project teachers differed from comparison classrooms 
throughout the school year was in the amount of time students spent engaged in small-group 
collaborative activity. 

There are a number of possible reasons why we observed this time-of-year effect. First, one could argue 
that project teachers are among the most innovative teachers within their schools and were predisposed to 
become a part of the Multimedia Project. The project, therefore, might not be the cause of the difference 
in classroom processes. This interpretation is not consistent with case study and interview data, however, 
in which many project teachers describe how the project has changed their view and practice of teaching. 

Another possible interpretation of these data are that even among project teachers, there is a natural 
building of component skills for projects that takes place in the fall. In the fall, many project teachers use 
time to teach students research skills or how to use different multimedia software packages. The teachers 
are still focusing on projects, but their work requires much more direct, teacher-led instruction at this 
stage. 

One thir d possible reason why we have observed this time-of-year effect is the timing of the Multimedia 
Fairs. The fairs are held in the spring each year and motivate much of the activity of the project teachers 
and their students throughout the spring months. At the time of our observations in April, many of the 
project classrooms were in the middle of working on projects they would show at the fairs. These fairs, in 
turn, motivate students' attention to an external audience, since people from other schools and the 
community will see their work. 

Even with this school year effect, a convincing case can be made that project teachers are more likely to 
engage students in small-group collaborative activity, regardless of whether they are working on their 
multimedia projects. In some cases, it may be that small group work is part of the school's philosophy, 
and the emphasis on collaboration cannot be attributed solely to the work of the Multimedia Project (see 
Penuel, Cole, Korbak, & Jump, under review). Still, the success of previous student projects appears to 
contribute to teachers' eagerness to use collaborative learning as a tool to promote greater mastery of 
content and skill in working well with others. 

Overall, the results suggest that the project is meeting its objective of transforming classroom processes 
so that they become more student-centered, especially while students are engaged in project-based 
learning using multimedia. The results suggest a strong role for the projects themselves and for the 
Multimedia Fair in contributing to these changes, since differences between project and comparison 
classrooms are much more evident in the spring than in the fall. Indeed, it may be that events like the 
fairs, which provide concrete links between the classroom and other classrooms and the community, 
deserve a more important place as levers for changing classroom practice. 
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